Evolution of Assortative Mating in a Population Expressing Dominance

In this article, we study the influence of dominance on the evolution of assortative mating. We perform a population-genetic analysis of a two-locus two-allele model. We consider a quantitative trait that is under a mixture of frequency-independent stabilizing selection and density- and frequency-dependent selection caused by intraspecific competition for a continuum of resources. The trait is determined by a single (ecological) locus and expresses intermediate dominance. The second (modifier) locus determines the degree of assortative mating, which is expressed in females only. Assortative mating is based on similarities in the quantitative trait ('magic trait' model). Analytical conditions for the invasion of assortment modifiers are derived in the limit of weak selection and weak assortment. For the full model, extensive numerical iterations are performed to study the global dynamics. This allows us to gain a better understanding of the interaction of the different selective forces. Remarkably, depending on the size of modifier effects, dominance can have different effects on the evolution of assortment. We show that dominance hinders the evolution of assortment if modifier effects are small, but promotes it if modifier effects are large. These findings differ from those in previous work based on adaptive dynamics.